The Animat Path to Artificial General Intelligence

Stewart Wilson introduced the term animat for artificial animals and outlined the animat path to artificial intelligence. In this paper the animat path to artificial general intelligence is explored. A general computational model is proposed for animats living in dynamic block worlds, e.g. in the Minecraft environment. The model uses mechanisms for learning and decision-making that are common to all animats. Each animat has its own sets of needs, sensors, and motors. It also has its own memory structure that undergoes continuous development and constitutes the basis for decision-making. The goal of the decision-making is always to keep the needs as satisfied as possible for as long as possible. The learning mechanisms are of two kinds: (i) structural learning that adds and removes nodes and connections of the memory structure; (ii) a local version of multi-objective Q-learning. The animats of the model are autonomous and able to adapt to arbitrary previously unseen block worlds without any need for seed knowledge. They adapt by learning basic skills such as foraging, locomotion, navigation, and pattern recognition. 1 General intelligence In psychology, the term general intelligence refers to the way that a person’s performance on one psychometric task tends to correlate with her performance on other tasks [Spearman, 1904]. In artificial intelligence, the term tends to be used more broadly for versatile and autonomous agents [Legg, 2008]. A survey of performance measures relating to general intelligence in humans and artificial systems can be found in [Hernández-Orallo, 2016]. According to the physicist Pieter van Heerden [Heerden, 1968]: Intelligent behavior is to be repeatedly successful at satisfying one’s psychological needs in diverse, observably different, situations on the basis of past experience. Interpreted broadly, van Heerden’s characterization of intelligent behavior takes all types of needs in Dörner’s taxonomy into account: physiological, social, and cognitive [Dörner, 2001]. Note that van Heerden’s characterization applies to all animal species, not just humans. It is also general in the sense that it does not rely on human judgement, like the Turing test does; or on human artifacts, like standard IQ tests do. Moreover, van Heerden’s characterization harmonizes with an embodied view of intelligence, whether or not the distinction between body and mind is maintained. In fact, the ability of an animal to satisfy its needs depends on the body, the control system of the body, and the interplay between the two, to the extent that those notions can be meaningfully separated in the first place. The goal of artificial general intelligence is to take the step from “narrow” AI programs that are tailored for specific tasks or problem domains to general AI programs with intelligence “at the human level and beyond” [Pennachin and Goertzel, 2007]. Reinforcement learning and in particular Q-learning has been used in agents where the goal is to accumulate reward over time [Sutton and Barto, 1998]. In standard reinforcement learning, the reward signal is one-dimensional; in multiobjective reinforcement learning, it is multidimensional [Roijers et al., 2013]. Sometimes it is straightforward to reduce multidimensional reward signals into scalars, e.g. money of several currencies can be converted into money of a single currency. Sometimes it is harder, e.g. in the case of an animal that receives a reward signal with an energy and a water component. No amount of energy can compensate for a lack of water and vice versa. Certain agents have a set of needs and receive a (multidimensional) reward signal that measures changes in the status of those needs. Such homeostatic agents strive to keep several internal signals in certain intervals [Konidaris and Barto, 2006; Yoshida, 2017]. For homeostatic agents, the above-mentioned characterization of intelligent behavior by van Heerden essentially coincides with the so-called reinforcement learning hypothesis [Lettman, 2006]: Intelligent behavior arises from the actions of an individual seeking to maximize its received reward signals in a complex and changing world. Deep Q-learning combines deep networks with reinforcement learning [LeCun et al., 2015; Schmidhuber, 2015]. One of the most prominent examples of this method in the direction of general intelligence is the generic Atari-game player that learned to play 31 Atari games at super-human level [Mnih and others, 2015]. Although deep Q-learning has been groundbreaking, several issues remain problematic: avoiding catastrophic forgetting; enabling lifelong, one-shot, and transfer learning; reducing the need for large training volumes; and supporting logical reasoning [Harrigan, 2016]. For a discussion of some theoretical problems associated with deep Q-learning, see [Wang and Li, 2016]. Graph structures that develop gradually have been studied, e.g. in finite automata learning [Angluin, 1980], cascade correlation networks [Fahlman and Lebiere, 1990], and deep network cascades [Angelova et al., 2015]. Cognitive architectures, e.g. Soar [Laird, 2012], ACT-R [Anderson et al., 2004], and MicroPsi [Bach, 2015], are computer systems that attempt to model aspects of the human mind, including general intelligence. Agent architectures reflect a wider notion that includes systems for artificial intelligence that do not necessarily aim for biological realism, e.g. OpenCog [Goertzel et al., 2014], AERA [Nivel et al., 2013], and NARS [Wang and Hammer, 2015]. Animal intelligence has been studied extensively: e.g., in comparative psychology and artificial life. The objects of study in artificial life include artificial evolution, cellular automata, and particle swarm optimization [Langton, 1997; Tuci et al., 2016]. Stewart Wilson introduced the term animat for artificial animals via the following postulates, quoting from [Wilson, 1986]: • The animal exists in a sea of sensory signals. At any moment, only some signals are significant; the rest are irrelevant. • The animal is capable of actions (e.g., movement) which change these signals. • Certain signals (e.g., those attendant on consumption of food) or their absence (e.g., those relating to pain) have special status. • He acts, both externally and internally, so as approximately to optimize the rate of occurrence of the special signals. Wilson also outlined the animat path to AI, which seeks to create artificial intelligence by modeling animal intelligence [Wilson, 1990]. In this paper we explore the animat path to artificial general intelligence. Section 2 describes our strategy for constructing a general and autonomous computational model. Section 3 describes our constructed model. Section 4 presents the prototype implementation Generic Animat of the model and gives examples of how it learns and makes decisions in the context of foraging, locomotion, navigation, and concept formation. Section 5 discusses the scalability of the model and Section 6, finally, draws some conclusions. Figure 1: A Minecraft world with blocks of type “water”, “grass”, “sand”, etc. The proposed computational model is partly a continuation of our previous work [Bach, 2015; Nivel et al., 2013; Strannegård et al., 2015; Strannegård and Nizamani, 2016]. The mechanisms for local Q-learning and structural learning are novel to the best of our knowledge.